MIRACLE at ImageCLEFmed 2007: Merging Textual and Visual Strategies to Improve Medical Image Retrieval

This paper describes the participation of MIRACLE research consortium at the ImageCLEF Medical Image Retrieval task of ImageCLEF 2007. For this campaign, our challenge was to research on different merging strategies, i.e. methods of combination of textual and visual retrieval techniques. We have focused on the idea of performing all possible combinations of well-known textual and visual techniques in order to find which ones offer the best results in terms of MAP and analyze if the combined results may improve the individual ones. Our system consists of three different modules: the textual (text-based) retrieval module, which indexes the case descriptions to look for those descriptions which are more relevant to the text of the topic; the visual (contentbased) retrieval component, which provides the list of case images that are more similar to the topic images; and, finally, the merging module, which offers different operators (AND, OR, LEFT, RIGHT) and metrics (max, min, avg, max-min) to combine and rerank the outputs of the two previous subsystems. These modules are built up from a set of basics components organized in four categories: (i) resources and tools for both general-domain and medical-specific vocabulary analysis, (ii) linguistic tools for text-based information retrieval, (iii) tools for image analysis and retrieval, and (iv) ad-hoc tools for result merging and reranking. We finally submitted 50 runs. The highest MAP was obtained with the baseline text-based experiment in English where only stemming plus stopword removal is performed. Neither tagging with UMLS medical concepts nor merging of textual and visual results proved to be of value to improve the precision with regards to the baseline experiment. However, the most interesting conclusion is that experiments that use the OR operator obtain higher MAP values than those with the AND operator.